1. Field of the Invention
The present invention relates to a signal processing apparatus and a signal processing method for processing pixel signals corresponding to a plurality of respective pixels; a solid-state image capturing device, such as a CCD solid-state image capturing apparatus or a CMOS solid-state image capturing apparatus, and in particular, a solid-state image capturing apparatus which divides light into wavelengths using a color filter to obtain a color image signal so as to correspond to a human's sense of sight, for performing a photoelectric conversion on and capturing an image of image light from a subject, using the signal processing apparatus and the signal processing method; an electronic information device, such as a digital camera (e.g., a digital video camera or a digital still camera), an image input camera (e.g., a monitoring camera), a scanner, a facsimile machine, a television telephone device, and a camera-equipped cell phone device, which electronic information device includes the solid-state image capturing element as an image input device used in an image capturing section thereof; a control program in which processing procedures for allowing a computer to execute each of the steps of the signal processing method are described; and a computer-readable storage medium storing the control program.
2. Description of the Related Art
In the solid-state image capturing apparatus of this type, its signal output decreases when an image-capturing subject is dark, which makes noise relatively conspicuous. Thus, in order to obtain sufficient S/N even when an image-capturing subject is dark, obtainment of higher sensitivity for pixels and noise reduction become essential techniques.
In the meantime, high color reproducibility is also important in a color solid-state image capturing apparatus. However, the sensitivity to respective wavelengths is different between the human eyes and the pixels of solid-state image capturing apparatus. Thus, high color reproducibility is achieved by adjusting the deviation between the human eyes and the output of the pixels. While processing includes white balance and color matrix, as major techniques for the color adjusting, a major technique for adjusting sensitivity to wavelength is to perform color matrix processing to an output signal.
For example, in a solid-state image capturing apparatus for outputting RGB signals, by color matrix processing, the RGB image-capturing signals (sensor output) of the solid-state image capturing apparatus are multiplied by a 3×3 determinant as the following formula (1) for conversion to become RGB signals for display for display apparatuses and copying machines.
                              (                                                                      R                  ⁢                                      :                                    ⁢                                                                          ⁢                  display                                                                                                      G                  ⁢                                                                          ⁢                  display                                                                                                      B                  ⁢                                                                          ⁢                  display                                                              )                =                              (                                                                                M                                          R                      ->                      R                                                                                                            M                                          G                      ->                      R                                                                                                            M                                          B                      ->                      R                                                                                                                                        M                                          R                      ->                      G                                                                                                            M                                          G                      ->                      G                                                                                                            M                                          B                      ->                      G                                                                                                                                        M                                          R                      ->                      B                                                                                                            M                                          G                      ->                      B                                                                                                            M                                          B                      ->                      B                                                                                            )                    ⁢                      (                                                                                R                    ⁢                                                                                  ⁢                    image                    ⁢                                          -                                        ⁢                    capturing                                                                                                                    G                    ⁢                                                                                  ⁢                    image                    ⁢                                          -                                        ⁢                    capturing                                                                                                                    B                    ⁢                                                                                  ⁢                    image                    ⁢                                          -                                        ⁢                    capturing                                                                        )                                              (        1        )            
In the color matrix processing, a display signal consisting of R display, G display and B display is normally given by multiplying an RGB image-capturing signal of the sensor output consisting of R image-capturing, G image-capturing and B image-capturing by a 3×3 determinant, as the formula (1) above.
The one added to the same color, such as MR→R, is referred to as a diagonal component, and the one added to a different color, such as MR→G, is referred to as a non-diagonal component. The diagonal component is 1 or greater, and normally, most of non-diagonal components have a negative value.
Hereinafter, an explanation will be provided with a solid-state image capturing apparatus which outputs RGB signals; however, application can also be made to a solid-state image capturing apparatus which captures an image of complementary colors of the ROB signals.
FIG. 9 is a flowchart schematically illustrating an image processing operation of a conventional solid-state image capturing apparatus.
As illustrated in FIG. 9, after the performance of A/D conversion processing on an image-capturing signal from an image capturing sensor (step S101), various kinds of signal processing, such as black level, white balance, color interpolation and noise reduction, are performed (step S102). Further, after the performance of color matrix processing (step S103), signal processing such as contrast emphasizing and γ correction processing is performed (step S104).
In such a conventional manner, only the same color matrix processing is multiplied to the entire output signal of an image.
It becomes difficult for a human to sense colors when it gets dark, and such a human sees a view with fewer colors. However, in such a degree of brightness, although such a view appears to a human with less color, the same colors are added with the same color matrix as a photopic vision. This causes, not only decrease in the color reproducibility, but also increase in noise due to forcibly added colors.
In order to maintain the color reproducibility, it is necessary to adjust the non-diagonal component of the color matrix appropriately. In doing so, it is known that the color reproducibility can be increased by using the color matrix in accordance with the brightness of a light source or a subject. This fact is described in Reference 1.
FIG. 10 is a block diagram illustrating an exemplary structure of an image processing circuit of a conventional CCD solid-state image capturing apparatus disclosed in Reference 1.
As illustrated in FIG. 10, in an image processing circuit 101 of a conventional CCD solid-state image capturing apparatus 100, an analog pixel signal (CCD input) for one frame, which is read out from an image capturing sensor, is input into a first stage signal processing circuit 102. The first stage signal processing circuit 102 is provided with a pre-amplifier, a band restricting video filter and the like, and the analog pixel signal to be input is sample-held and predetermined signal processing, such as amplification processing, is performed thereon.
The analog pixel signal processed in the first stage signal processing circuit 102 is converted into a digital pixel signal by an analog/digital (A/D) converter 103, and the digital pixel signal is input into a color dividing circuit 104. In the color dividing circuit 104, color dividing processing is performed on the digital pixel signal having respective color components of color chip filters of complementary colors, the digital pixel signal being converted into an RGB signal consisting of respective colors of red (R), green (G) and blue (B). The RGB signal is input into a color matrix circuit 105 from the color dividing circuit 104.
In the color matrix circuit 105, arithmetic processing for multiplying a predetermined matrix coefficient is performed on the ROB signal divided in the color dividing circuit 104, so that the signal is converted into an ROB signal having an appropriate color balance. The ROB signal provided with the arithmetic processing by the color matrix circuit 105 is stored in an image memory 106.
The RGB signal stored in the image memory 106 is read out as appropriate and is input into a latter stage signal processing circuit 107. The latter stage signal processing circuit 107 has a color balance amplifier. By the color balance amplifier, the color balance of the RGB signal is converted based on white balance data being input from a system controller 112. Further, in the latter stage signal processing circuit 107, image processing, such as clamp, γ correction, contour emphasizing and character impose, is performed on the RGB signal with a converted color balance. The RGB signal on which the image processing has been performed by the latter stage signal processing circuit 107 is converted into an analog signal by a digital/analog (D/A) converter 108. The analog RGB signal goes through video signalizing processing in an output control section 109 having a cable driver and an encoder, and is sent to a TV monitor. Thereby, an image of a subject captured by a color CCD is reproduced on a display of a TV monitor.
The RGB signal stored in the image memory 106 is also output to a light adjustment controlling section 110. In the light adjustment controlling section 110, a luminance signal is generated from the RGB signal. Further, based on the luminance signal, a light adjustment control signal I used for light amount adjustment of illumination light is calculated. The light adjustment control signal I is obtained by calculating an average value of luminance signals of all the pixels constituting an effective area displayed on a display of a TV monitor of an image capturing area of a CCD image sensor, and a peak value of luminance signals of pixels constituting a center area of the effective area, and by weighting respective values.
The light adjustment control signal I calculated by the light adjustment controlling section 110 is output to a light adjustment section 111. In the light adjustment section 111, a lens opening is driven based on the light adjustment control signal I to adjust the amount of light passing through the lens opening. As a result, the amount of white light, which is output from a light source section and enters an incident end surface of a light guide, is adjusted. Thereby, the image of the subject reproduced on a display of a TV monitor has the most appropriate luminance. The light adjustment control signal I is also output to a system controller 112, and is used for processing of converting a color matrix coefficient, which will be described later.
As such, the light adjustment control signal I from the light adjustment controlling section 110 is obtained by the system controller 112 and a color matrix coefficient is selected in accordance with the luminance of a subject. This will be explained with reference to FIG. 11.
FIG. 11 is a flowchart illustrating processing procedures of a color conversion adjustment by the system controller 112 in FIG. 10.
As illustrated in FIG. 11, color conversion is performed in a color matrix circuit 105 using a default color coefficient, and a light adjustment control signal I is calculated in the light adjustment controlling section 110 based on the image signal after the conversion. The light adjustment control signal I calculated in the light adjustment controlling section 110 is input (step S1), and thereafter, advancing to the processing of step S2, the light adjustment control signal Z is compared with a predetermined threshold. This threshold is set at a level, at which it is not problematic to determine that a subject image is more bright than usual when the light adjustment control signal I is more than the threshold and the image reproduced on a display of a TV monitor becomes highly bright. For example, a maximum high luminance value at which a subject image does not cause halation is determined as a threshold, for a reason to be described later.
When the light adjustment control signal T is smaller than the threshold at the processing in step S2, that is, when a subject image is not highly bright, the procedure advances to the processing in step S3. In step S3, a first color matrix coefficient is selected. On the other hand, when the light adjustment control signal I is larger than the threshold at the processing in step S2, that is, when a subject image is highly bright, the procedure advances to the processing in step S4. In step S4, a second color matrix coefficient is selected. In the first and second color matrix coefficient α, a value of at least one element (e.g., element a21) in a matrix as shown in the following formula (2) is different from each other. That is, the first color matrix coefficient is defined as the one corresponding to standard coloring (normal coloring), and the second color matrix coefficient, in which at least one element value is different from the first color matrix coefficient, is determined such that the coloring of the subject image changed in the threshold is corrected to the standard coloring. A specific manner for determining a second color matrix coefficient will be described later.
                    α        =                  [                                                                      a                  ⁢                                                                          ⁢                  11                                                                              a                  ⁢                                                                          ⁢                  12                                                                              a                  ⁢                                                                          ⁢                  13                                                                                                      a                  ⁢                                                                          ⁢                  21                                                                              a                  ⁢                                                                          ⁢                  22                                                                              a                  ⁢                                                                          ⁢                  23                                                                                                      a                  ⁢                                                                          ⁢                  31                                                                              a                  ⁢                                                                          ⁢                  32                                                                              a                  ⁢                                                                          ⁢                  33                                                              ]                                    (        2        )                                          [                                                    R                                                                    G                                                                    B                                              ]                =                  α          ⁡                      [                                                                                R                    ′                                                                                                                    G                    ′                                                                                                                    B                    ′                                                                        ]                                              (        3        )            
Upon selection of a color matrix coefficient at step S3 or S4, the procedure advances to step S5, and a control instruction for instructing to execute arithmetic processing by the formula (3) described above using the selected color matrix coefficient α is output to the color matrix circuit 105.
In the color matrix circuit 105, the conversion processing of the RGB signal is performed based on the formula (3). In the formula (3), R′ indicates a red color after color division, G′ indicates a green color after color division, and B′ indicates a blue color after color division. R indicates a red color after color conversion, G indicates a green color after color conversion, and B indicates a blue color after color conversion. Further, α is a color matrix coefficient, which is a matrix consisting of 3×3 elements as represented by the aforementioned formula (2).
Reference 1: Japanese Laid-Open Publication No. 2004-194993